Read the original article here.

---

Operating Systems: The Foundation for Building Computers

When embarking on the journey of building a computer from scratch, one quickly realizes that hardware alone is insufficient. A crucial layer of software is needed to make the hardware usable and manage its resources effectively. This foundational software is the Operating System (OS). Understanding the role and components of an OS is essential for anyone wanting to build, understand, or program computers at a fundamental level.

An operating system is the system software that acts as the intermediary between the user/application software and the computer hardware. It manages the computer's resources and provides common services for application programs.

The Essential Role: Why Operating Systems Exist

Defining an operating system precisely can be challenging, but it's often described as "the layer of software that manages a computer's resources for its users and their applications."

The operating system includes the core software that is always running, known as the kernel. Other essential programs that interact closely with the OS but may not be part of the kernel are called system programs. All other software the user runs is generally referred to as applications.

Operating systems fulfill three primary purposes:

1. Resource Allocation: An OS allocates the computer's resources among different applications and users. Resources include the Central Processing Unit (CPU) time, memory space, storage space, and peripherals like printers or network interfaces. The OS decides when applications get to use the CPU, how much memory they can access, and manages access to shared devices. This prevents one program from monopolizing the system and ensures fair usage. On modern computers, users often run multiple applications simultaneously; the OS manages this concurrency. It also isolates applications from each other to protect them from errors or security vulnerabilities in other programs, while providing mechanisms for secure communication between them when necessary.

2. Hardware Abstraction: The OS provides a simplified interface for accessing hardware details. Instead of programmers needing to write code that directly manipulates specific hardware registers or memory addresses (which vary greatly between different computer models), they interact with the OS through defined system calls. The OS handles the complex, low-level interactions with the hardware. This abstraction makes it easier to write application software and allows applications to run on different hardware without extensive rewriting, provided the OS is available on that hardware. Techniques like virtualization, including virtual memory, create the illusion of more resources than physically exist, simplifying programming and resource management.

3. Common Services: Operating systems provide standard services that applications frequently need, such as accessing files on storage devices, managing network connections, or displaying information on the screen. By providing these services, the OS prevents each application from having to implement them from scratch, reducing complexity and development time for application programmers. The range of services offered varies widely between operating systems, but they constitute a significant portion of the OS code.

Building Blocks of an Operating System: Core Components

To understand how an OS works from a low-level perspective, like when building hardware or writing a basic OS, it's crucial to examine its key components and how they interact with the hardware. Modern operating systems ensure that virtually all user software interacts with the OS to access hardware resources.

The Kernel

The kernel is the core of the operating system. It is the part of the OS that is always resident in memory and has complete control over the system.

Kernel: The central, fundamental part of a computer operating system that manages system resources (like CPU, memory, I/O devices) and provides essential services for applications. It is the bridge between application software and the hardware.

A primary function of the kernel is to provide protection between different applications and users. This is critical for system reliability (isolating errors) and security (limiting malicious software, protecting data). Most operating systems implement two main modes of operation, often with hardware support:

• User Mode: In this mode, application programs run. The hardware enforces restrictions, preventing applications from executing certain privileged instructions or accessing memory/hardware resources they are not allowed to.
• Kernel Mode (or Supervisor Mode): The kernel runs in this mode. It has unrestricted access to all hardware, memory, and system instructions. Any attempt by user-mode software to perform a privileged operation (like accessing a hardware device directly or changing memory protection settings) must transition into kernel mode, typically via a system call or an interrupt.

The kernel is responsible for:

• Managing memory for all processes.
• Controlling access to I/O devices.
• Scheduling processes/threads to run on the CPU.
• Handling system calls from applications.
• Responding to hardware interrupts.

Program Execution and Processes

Running an application program isn't just loading code into memory and jumping to the first instruction. The operating system plays a vital role.

Process: An instance of a computer program that is being executed. A process includes the program code, its current activity, its data, its allocated resources (memory, files, etc.), and the state of its CPU registers.

When you launch a program, the OS kernel typically performs the following steps:

1. Creation of a Process: The OS creates a new process to contain the running program.
2. Resource Allocation: It allocates system resources for the process, including a dedicated memory space (its address space).
3. Loading Code: The program's executable code is loaded from storage (like a hard drive) into the allocated memory space.
4. Initialization: The OS sets up the process's initial state, including its CPU registers, stack, and other data structures needed by the OS to manage the process.
5. Scheduling: In a multitasking system, the OS places the process into a queue of runnable processes and assigns it a priority.
6. Execution Start: The OS initiates execution of the program by setting the CPU's program counter to the program's entry point within the allocated memory.

Once the program starts executing, it interacts with the OS and hardware primarily through system calls.

System Call: A programmatic way in which a computer program requests a service from the kernel of the operating system. This is the standard interface between the application and the OS kernel. Examples include reading/writing files, creating a new process, allocating memory, or sending data over a network.

While application code usually runs directly on the hardware in user mode, it must request privileged operations or access to shared resources via system calls, which cause a transition into kernel mode.

Some systems allow one application to request the OS execute another application within the same process, either as a subroutine call or in a separate thread. This leads to the concept of concurrency, discussed later.

Handling Events: Interrupts

Interrupts are a fundamental mechanism that allows the operating system and hardware to react efficiently to events occurring in the system's environment, whether from hardware devices or software.

Interrupt: A signal indicating an event that requires immediate attention from the CPU. It causes the CPU to suspend its current execution, save its state, and transfer control to a special routine called an interrupt handler or interrupt service routine (ISR).

Interrupts are crucial because they allow the CPU to perform other tasks instead of constantly checking (polling) if a device needs attention or if a significant software event has occurred. When an interrupt occurs, the CPU's normal flow of execution is disrupted, and control is transferred to the OS's interrupt handler for that specific event.

The process of handling an interrupt involves several steps, supported by both hardware and the operating system:

1. Interrupt Request: A device (hardware interrupt) or instruction (software interrupt) signals the CPU.
2. CPU Acknowledgment: The CPU finishes its current instruction, acknowledges the interrupt, and determines which type of interrupt occurred (often via an interrupt vector provided by the hardware).
3. State Saving: The CPU hardware or the initial part of the OS interrupt handler saves the state of the currently running process or thread. This includes saving the program counter (so the OS knows where to resume later), status registers, and other relevant CPU registers onto the call stack or into a system table.
4. Transfer Control: The CPU jumps to the address of the appropriate Interrupt Service Routine (ISR) in kernel mode.
5. Service the Interrupt: The ISR executes in kernel mode to handle the event (e.g., read data from a keyboard, handle a memory access error, perform a context switch).
6. State Restoration: After the ISR finishes, the OS restores the previously saved state of the interrupted process.
7. Return: The CPU returns control to the interrupted process, resuming its execution from where it left off.

Servicing an interrupt often requires a context switch, especially if the interrupt causes the currently running process to wait (e.g., for I/O) or if a higher-priority task is now ready to run.

Context Switch: The process by which a CPU switches from one process or thread to another. It involves saving the state (context) of the current process/thread and loading the saved state of the next process/thread to be executed.

Hardware Interrupts

Hardware interrupts are generated by peripheral devices to signal that they need attention or have completed an operation.

Hardware Interrupt: An event signaled by a hardware device to the CPU, requiring the CPU to stop its current task and handle the device's request.

Since hardware devices like disk drives, network cards, keyboards, and mice operate at speeds much slower than the CPU, waiting for them using techniques like polling (where the CPU repeatedly checks the device status) would waste vast amounts of CPU time. Interrupts allow the CPU to continue executing other tasks and only attend to the device when it signals it's ready (e.g., data is available from the network card, a key has been pressed).

For high-speed devices like hard drives, interrupting the CPU for every byte or word of data transferred would still be inefficient. This led to the development of Direct Memory Access (DMA).

Direct Memory Access (DMA): A hardware feature that allows peripheral devices to transfer data directly to or from main memory without involving the CPU. The CPU initiates the transfer, and the DMA controller handles the bulk data movement, interrupting the CPU only when the entire transfer is complete.

DMA significantly reduces the CPU overhead for I/O operations.

Example: A Detailed Look at a Block I/O Write with DMA

Let's trace what happens when an application program requests to write a block of data to a storage device using a system call, leveraging DMA and interrupts:

1. Application Calls System Call: The application executes a system call (e.g., write(file_descriptor, buffer, count)) to write data.
2. Transition to Kernel Mode: The system call traps into the kernel. The CPU switches from user mode to kernel mode.
3. Kernel Prepares for I/O:
   • The kernel saves the state (context) of the currently running application process (e.g., in its Process Control Block - PCB).
   • It finds the appropriate device driver for the storage device.
   • It prepares the data to be written in a kernel buffer in memory.
   • It sets up the DMA controller by giving it the memory address of the data buffer and the size of the data block (count). It also tells the DMA controller where on the device the data should be written.
   • It updates an internal data structure, like a device-status table, noting that this process is waiting for this specific I/O operation to complete. The process's PCB address might be stored here.
4. Initiate Hardware Operation: The kernel issues a command to the storage device hardware (often through the DMA controller) to begin the write operation.
5. Context Switch Away: Since the write operation will take a relatively long time, the kernel marks the calling process as waiting (not runnable) and performs a context switch to schedule another process to run on the CPU from the ready queue. The original process is now suspended, waiting for the I/O to finish.
6. DMA Transfer: The DMA controller and the storage device hardware work together to transfer the data block directly from the kernel buffer in memory to the device. The CPU is not involved in this data transfer itself.
7. I/O Completion Interrupt: Once the device has finished writing the data block, the storage device or the DMA controller generates a hardware interrupt signal to the CPU.
8. CPU Interrupt Handling (Entry):
   • The CPU stops executing the currently running process.
   • It saves the state (PC, status registers, other registers) of this currently running process onto its stack or in a system area.
   • It switches to kernel mode (if not already there).
   • It determines the type of interrupt, often by reading a value placed on the data bus by the device (the interrupt vector number).
9. Kernel Interrupt Service Routine (ISR) Executes:
   • The CPU jumps to the specific ISR address corresponding to the storage device completion interrupt.
   • The ISR identifies which I/O request has completed, often by consulting the device-status table using information provided by the interrupt (e.g., the device number).
   • It finds the waiting process associated with this completed request (using the stored PCB address).
   • It updates the device-status table to indicate the operation is complete.
   • It changes the state of the waiting process from waiting back to ready (runnable).
   • It might place this process back into the ready queue, potentially at a higher priority or making it eligible to run during the next scheduling decision.
10. Context Switch Back (Later): When the OS scheduler decides to run the now-ready process again (either immediately if it was high priority, or when its turn comes in the ready queue):
    • The kernel performs a context switch.
    • It restores the saved state (registers, PC, etc.) of this process from its PCB and stack.
    • The CPU switches back to user mode.
11. Resumption of Process: The process resumes execution from the instruction after the original system call that initiated the write. From the application's perspective, the write call has now completed.

This detailed example highlights the interplay between hardware interrupts, DMA, context switching, and kernel data structures, which are core concepts in low-level OS design.

Software Interrupts and Signals

While hardware interrupts are triggered by devices, software interrupts are generated by software events.

Software Interrupt: An event generated by a software instruction or condition that causes the CPU to switch control from the current program to an operating system routine. Also known as a trap, exception, or fault.

Software interrupts can be triggered intentionally by an instruction (like the INT instruction on x86 processors, used to invoke system calls or other kernel services) or unintentionally by error conditions (like division by zero, accessing an invalid memory address – a segmentation fault or page fault, discussed later). They function similarly to hardware interrupts in that they cause a mode switch to the kernel and jump to a specific handler routine.

In Unix-like operating systems, a specific form of software interrupt or asynchronous notification between processes is called a Signal.

Signal: A limited form of inter-process communication used in Unix-like systems to notify a process of an event. Signals are asynchronous; they can be delivered to a process at almost any time during its execution.

Signals are software interrupts delivered to a specific process rather than the CPU directly. They inform the process that an asynchronous event has occurred.

Example: Using Signals in Unix-like Systems

• Invoking System Calls: Traditionally, the INT assembly instruction on x86 (or similar instructions on other architectures) was used to trigger a software interrupt corresponding to a specific system call number, causing the kernel to execute the requested service. Modern systems often use instructions like syscall or sysenter which are optimized for this purpose, but the principle is similar – transition to the kernel to request a service.
• Error Handling: A division-by-zero error generates a specific type of fault (a software interrupt). The kernel's handler for this fault will typically terminate the offending process and report the error.
• User Interaction: Pressing Control+C in a terminal usually generates a SIGINT (Interrupt Signal) that is sent to the foreground process, typically causing it to terminate.
• Inter-Process Communication (IPC): Processes can send signals to each other using system calls like kill(pid, signum). The receiving process's signal handler (a function registered with the OS) is executed asynchronously when the signal is delivered. This can be used to coordinate tasks. For instance, in a pipeline alpha | bravo, the kernel uses signals to notify bravo when alpha has written data to the pipe and bravo can proceed to read.

Signals can indicate various events, including:

• A process finishing normally or abnormally.
• An error exception (e.g., illegal instruction, invalid memory access).
• Running out of a system resource.
• An alarm event (timer expiring).
• Being aborted from the keyboard (Ctrl+C).
• Tracing alerts for debugging.

Managing Memory

One of the most critical functions of the OS kernel in a multitasking environment is managing the computer's memory (RAM).

Memory Management: The function of an operating system that handles primary memory. It tracks every memory location, decides which processes get memory, allocates memory upon request, and reclaims memory when it is no longer needed.

The goal is to ensure that programs have the memory they need, prevent them from interfering with each other's memory, and efficiently utilize the limited physical RAM.

• Cooperative Memory Management: Used in very early multitasking systems (like some versions of Windows prior to Windows 95), this model relied on programs voluntarily respecting the memory allocated to them by the OS and not attempting to access memory belonging to others. This approach is highly unreliable; a single buggy or malicious program could easily overwrite critical data belonging to other programs or the OS itself, leading to system crashes. This model is essentially obsolete for general-purpose computing.
• Protected Memory: Modern operating systems use protected memory. This requires hardware support, often through a Memory Management Unit (MMU) on the CPU or integrated into the processor design.

Memory Protection: The OS kernel and hardware working together to control access to memory locations, preventing one process from accessing memory that has not been allocated to it.

Mechanisms like segmentation and paging are used to implement memory protection. They divide memory (or the address space visible to a program) into blocks (segments or pages). Special hardware registers (often managed by the OS kernel when running in kernel mode) define the valid memory ranges that the currently running process is allowed to access while in user mode.

If a user-mode program attempts to access a memory address outside its allowed range:

1. The MMU hardware detects the invalid access.
2. The MMU triggers a specific type of software interrupt, often called a segmentation fault or page fault (depending on the memory protection mechanism used).
3. The CPU switches to kernel mode and jumps to the OS's handler for this fault.
4. The kernel determines which process caused the fault and which memory address was accessed.
5. Because accessing arbitrary memory is usually a sign of a programming error or a malicious attempt, the kernel typically terminates the offending program to protect the rest of the system.

This hardware-assisted memory protection is fundamental to the stability and security of modern operating systems. Systems like Windows 3.1 had limited protection that was easy to bypass, contributing to frequent crashes (often seen as "General Protection Faults").

Virtual Memory

Virtual memory is a technique that relies on memory protection mechanisms (like paging) to provide a program with the illusion of having a larger, contiguous block of memory than is physically available in RAM.

Virtual Memory: A memory management technique where the operating system abstracts physical memory, allowing processes to use a large, consistent range of memory addresses (the virtual address space) which may be mapped to non-contiguous physical memory locations or even storage devices.

Here's how it works conceptually:

1. Each process is given its own virtual address space. These virtual addresses are what the program uses internally (e.g., when using pointers).
2. The MMU hardware, guided by tables set up by the OS kernel, translates these virtual addresses into physical memory addresses (the actual addresses in the RAM chips).
3. The OS kernel controls these mapping tables. Not all of a process's virtual address space needs to be mapped to physical RAM at any given time. Parts of it can be temporarily stored on a slower storage device, like a hard drive or SSD (this is called the swap space or paging file).
4. If a program tries to access a virtual memory address that is allocated to it but is not currently mapped to physical RAM (i.e., it's been swapped out to disk), this triggers a specific type of page fault interrupt.
5. The kernel's page fault handler intercepts this interrupt. It recognizes that the access is valid within the program's allocated virtual space but the required data is not in RAM.
6. The kernel finds the data on the swap device, reads it back into physical RAM (potentially moving another less-used page of memory out to swap to make room), updates the MMU's mapping tables to point the virtual address to the new physical location, and then allows the interrupted instruction to resume execution.

This mechanism allows the OS to run more programs than would fit entirely in physical RAM, improves memory utilization, and simplifies memory management from the programmer's perspective.

Handling Multiple Tasks: Concurrency

Modern computers can seemingly run multiple programs at the same time. The operating system manages this concurrency.

Concurrency: The ability of an operating system or program to deal with multiple tasks apparently at the same time. On a single-core CPU, this is achieved through rapid switching between tasks (multitasking). On multi-core CPUs, it can involve true simultaneous execution (parallelism).

• Processes vs. Threads:

  • A process is a self-contained instance of a running program with its own independent memory space and resources. Creating a new process involves significant overhead for the OS (allocating memory, setting up data structures).
  • A thread is a smaller unit of execution within a process. Multiple threads within the same process share the same memory space and resources, but each has its own program counter, stack, and set of registers. Creating a thread is generally faster and requires less overhead than creating a process.

• Multitasking and Scheduling: The OS kernel is responsible for scheduling processes and threads to run on the available CPU core(s). It decides which task gets to run next and for how long.

  • Cooperative Multitasking: (Older, less common) Processes voluntarily yield control of the CPU back to the OS when they are ready or when they need to wait (e.g., for I/O). A single misbehaving process that never yields can hang the entire system.
  • Preemptive Multitasking: (Modern) The OS kernel uses hardware timers (interrupts) to periodically take control away from the currently running process/thread, even if it's not ready to yield. This allows the OS to enforce time limits and ensure fair allocation of CPU time among all runnable tasks, preventing any single task from monopolizing the CPU.

• Context Switching (Revisited): Preemptive multitasking relies heavily on context switching. When the OS decides to switch from one thread/process to another:

  1. It saves the current state of the running task (its registers, program counter, stack pointer, etc.) into a data structure (like a Thread Control Block or Process Control Block).
  2. It loads the saved state of the next task to be run into the CPU's registers.
  3. The CPU then resumes execution of the new task from where it last left off.

Context switching has overhead, as it takes time to save and restore state. Threads within the same process generally have less context to save/restore than full processes, making thread switching faster.

• Parallelism: On systems with multiple CPU cores, the OS scheduler can run multiple threads or processes simultaneously on different cores. This enables true parallelism and can significantly speed up programs designed to be multi-threaded.

Storing Data Permanently: The File System

Computers need to store data permanently, even when the power is off. This is the role of non-volatile storage devices like hard disk drives (HDDs) and solid-state drives (SSDs). Unlike RAM, which is accessed directly by the CPU, these devices require a more complex interface and data organization.

File System: An operating system component that manages how data is stored, retrieved, and organized on storage devices like hard drives or SSDs. It provides an abstraction layer over the raw hardware sectors and blocks, presenting data to users and applications as named files and directories (folders).

Key functions of a file system provided by the OS:

• Abstraction: Presents data as logical units (files) with human-readable names, rather than raw physical block numbers on the disk.
• Organization: Allows users and applications to organize files into hierarchical directories. The OS keeps track of the directory structure and file locations.
  • An absolute path specifies a file's location starting from the root directory (e.g., /home/user/document.txt on Linux, C:\Users\User\Document.txt on Windows).
  • A relative path specifies a file's location relative to the current working directory (e.g., document.txt if you are already in /home/user/).
• Access Control: Manages permissions, determining which users or processes can read, write, or execute files. (Often linked to the OS security features).
• Management of Free Space: Tracks which areas on the storage device are available to store new data.
• Performance: Uses techniques like caching (keeping frequently accessed data in faster memory) and prefetching (reading data the application is likely to need next) to improve access speed.
• Reliability and Data Integrity: Implements mechanisms to protect data from corruption or loss due to hardware failures or system crashes.

Applications interact with the file system through OS system calls for operations like:

• Creating, deleting, opening, and closing files.
• Reading data from or writing data to files.
• Listing files in a directory.
• Changing file permissions.

Internal Structure of a File System (Conceptual):

While abstracted away from the user, the OS file system component needs to keep track of where files and directories are physically stored on the disk. This involves several structures:

1. Directory Entries: Directories contain entries that map human-readable filenames to a unique identifier for the file (often called an inode number in Unix-like systems).
2. File Metadata (e.g., Inodes): The OS needs to store information about each file, such as its size, ownership, permissions, creation/modification dates, and crucially, the location(s) of the data blocks on the disk that contain the file's content. This information is often stored in structures indexed by the file identifier.
3. Data Blocks: The actual file content is stored in fixed-size or variable-size blocks on the storage device.
4. Indexing Structure: To find the data blocks associated with a file's metadata, the OS uses an indexing structure (often a tree-like structure) that maps the file identifier to the list or location of its data blocks.
5. Free Space Map: A mechanism (like a bitmap or linked list) to track which data blocks on the storage device are currently not being used and are available for new data.

When a file is saved, the OS allocates free blocks, updates the free space map, writes the data to the blocks, and updates the file's metadata and the directory entry. When a file is read, the OS looks up the filename in the directory, finds the file metadata, uses the index to find the data block locations, and reads the data from the device into memory.

Ensuring Reliability:

File systems employ techniques to protect data:

• Atomic Operations: File writing protocols are designed such that critical updates (like writing metadata and the actual data) are done in a sequence that can be rolled back or recovered if a crash occurs mid-operation, preventing the file system from being left in an inconsistent state. Journaling file systems are a common implementation.
• Redundancy: Techniques like RAID (Redundant Array of Inexpensive Disks) store data across multiple drives in a way that allows recovery even if one drive fails.
• Checksums: Mathematical checks can be performed on data blocks to detect if they have been corrupted. The OS or hardware can sometimes use redundant copies to correct errors.

Protecting the System: Security

Operating system security is paramount in a world where computers are connected to networks and store sensitive data.

Operating System Security: The mechanisms and policies implemented by the OS to protect the system's resources (data, hardware, software) from unauthorized access, modification, or denial of service. The primary goals align with the CIA Triad: Confidentiality (preventing unauthorized disclosure), Integrity (preventing unauthorized modification), and Availability (ensuring resources are accessible to authorized users when needed).

Key principles and techniques used in OS security:

1. Isolation: Keeping different security domains separate is fundamental. In an OS context, these domains include the kernel itself, individual processes running applications, and potentially virtual machines running entire OS instances. The kernel isolates user processes, and hypervisors isolate virtual machines. This limits the damage a breach in one domain can cause to others.
2. Least Privilege: Granting each process, user, or component only the minimum set of permissions and resources necessary to perform its function and nothing more.
3. Access Control: Controlling who or what can access specific resources (files, devices, network ports). This is often implemented using Access Control Lists (ACLs), which list permissions for users or groups on objects.
4. Privilege Separation: Designing software such that different functions requiring different privilege levels are separated into different components (e.g., a network server might have a small privileged component that handles binding to a low-numbered port and a larger, unprivileged component that handles client connections).
5. Minimizing Attack Surface: Designing the OS and its components to be as simple as possible, exposing only necessary functionality and reducing the amount of code that is accessible to potential attackers.
6. Checking All Requests: The OS must validate all requests from user-mode processes before performing privileged operations or granting access to resources.

Security and OS Architecture:

The architecture of the OS kernel can influence its security:

• No Isolation: Very old or extremely simple systems might have no protection between applications or between applications and the kernel. Highly insecure.
• Monolithic Kernel: Most general-purpose OSes (like Linux, Windows) use this design. The kernel runs as a single, large program in kernel mode. If any part of the kernel has a vulnerability, the entire kernel (and thus the entire system) can be compromised.
• Microkernel: This design moves many OS services (like file systems, network stacks, device drivers) out of the main kernel space and runs them as separate processes (often in user mode) that communicate with a minimal kernel core. A vulnerability in a file system process, for example, might not compromise the entire kernel, potentially improving security.
• Unikernel: An extreme approach for specialized applications (often in cloud or embedded environments). The application is linked directly with a minimal set of OS libraries (a "library OS") to create a single, highly specialized address space image. There is no separation between application and OS code, but since it runs a single application and is highly customized, the attack surface is drastically reduced.

Vulnerabilities and Mitigation:

Operating systems are complex software, and bugs are inevitable, some of which can be security vulnerabilities.

• Code Vulnerabilities: Many OSes are written in languages like C or C++, which, while powerful, are susceptible to vulnerabilities like buffer overflows (where writing past the end of a buffer can overwrite other data or executable code) due to a lack of automatic bounds checking.
• Hardware Vulnerabilities: Flaws in CPU design or implementation (like those exploited by Spectre or Meltdown) can sometimes be used to bypass OS security measures.
• Malicious Code: Deliberate backdoors or malicious components could be inserted.

OS developers use various techniques to improve security and mitigate vulnerabilities:

• Hardening: Configuring the OS and its services to reduce the attack surface and enable security features.
• Security Features: Implementing Address Space Layout Randomization (ASLR), Data Execution Prevention (DEP/NX), Control-Flow Integrity (CFI), mandatory access controls (MAC), etc.
• Code Review and Testing: Rigorous checking of code for flaws.
• Transparency (Open Source): For open-source operating systems like Linux, the source code is publicly available. This allows a large community to review the code, potentially finding and fixing vulnerabilities faster. Andrew S. Tanenbaum argues that releasing source code prevents developers from relying on "security by obscurity" (hoping attackers won't find vulnerabilities because the code is secret), forcing them to design systems that are inherently more secure.

Interacting with the User: User Interface

The operating system provides a way for humans to interact with the computer. This is the User Interface (UI).

User Interface (UI): The means by which a user interacts with a computer system, including the input methods (keyboard, mouse, touch) and output methods (screen display, sound).

The two most common types of user interfaces provided by OSes are:

• Command-Line Interface (CLI): The user interacts by typing commands on a text-based console. The OS processes the command and typically displays text output. CLIs are powerful, scriptable, and often preferred by developers and system administrators for their precision and efficiency, but they require memorizing commands. Examples: Bash (Linux), Command Prompt (Windows), PowerShell (Windows).
• Graphical User Interface (GUI): The user interacts using visual elements like windows, icons, menus, buttons, and pointers (WIMP paradigm). Input is typically via mouse, keyboard, or touch. GUIs are generally considered more intuitive and user-friendly for typical tasks and are dominant on personal computers and smartphones. Examples: Windows Desktop, macOS Aqua, various Linux desktop environments (GNOME, KDE), Android UI, iOS UI.

The OS includes the necessary software (display servers, window managers, UI toolkits) and drivers for input devices (keyboard, mouse, touchscreen) to support these interfaces. Building a functional GUI stack is significantly more complex than implementing a simple text-based CLI.

Different Flavors: Types of Operating Systems

Operating systems are designed for various purposes and hardware configurations. Understanding these types shows how the core OS concepts are adapted to different needs.

• Multicomputer Operating Systems: Designed for systems consisting of multiple independent computers (nodes), each with its own CPU and memory, connected by a network. These are common in large computing clusters or cloud environments. The OS or additional middleware manages communication (message passing) between nodes and might provide features like distributed shared memory or remote procedure calls. Performance often depends on minimizing inter-node communication overhead.
• Distributed Systems: A broader term for a collection of networked computers that appear to the user as a single, coherent system. Each computer might run its own OS and file system. Middleware is often used on top of the local OSes to provide services like distributed file systems or consistent resource access across the network. Unlike multicomputers, these nodes can be geographically dispersed.
• Embedded Operating Systems: Specialized OSes designed for embedded computer systems found in devices like household appliances, cars, industrial machinery, or IoT devices. They are typically small, resource-constrained, and designed for specific hardware and a fixed set of applications. Often, they do not allow users to install new software, which simplifies security and design by reducing the need for protection between arbitrary applications. Examples: Embedded Linux, QNX, VxWorks, RTOS variants.
• Real-time Operating Systems (RTOS): Critical for systems where operations must happen within strict time constraints.
  • Hard Real-time: Guarantees that critical tasks will be completed exactly on time. Used in applications where missing a deadline would cause catastrophic failure (e.g., avionics, medical devices, industrial control). Often very minimal, sometimes resembling just a library of OS functions linked directly with the application.
  • Soft Real-time: Prioritizes tasks to meet deadlines but can tolerate occasional misses. Used where timeliness is important but not absolutely critical (e.g., multimedia playback, smartphones).
• Hypervisor: An OS-like layer that runs virtual machines (VMs). Each VM acts as if it has its own dedicated hardware and runs its own guest OS. The hypervisor manages the physical hardware and allocates resources to VMs, isolating them from each other. This is crucial for cloud computing and server virtualization. VMs can be paused, saved (snapshots), and moved, offering flexibility for development, testing, and deployment.
• Library Operating System (LibOS) / Unikernel: An approach where typical OS services (networking, file system) are provided as libraries that are compiled and linked directly with a single application. This creates a specialized, single-address-space machine image (a unikernel) containing only the necessary code. Unikernels eliminate the overhead of context switches between user and kernel modes because everything runs in a single privilege level. They offer potential benefits in terms of size, boot time, performance, and security (due to a minimal attack surface) but are less flexible than general-purpose OSes as they run only one application.

A Journey Through Time: History of Operating Systems

Understanding the history of OS development provides context for why systems are designed the way they are today and the problems earlier systems faced.

• The Beginning (Late 1940s - Early 1950s): No Operating System: Early computers were programmed directly using physical wiring (plugboards) or machine code entered via switches or punched cards. There was no intermediate software layer; programmers interacted directly with the hardware. Running a new program required significant manual setup by operators.
• Early Batch Systems (Mid-1950s): With the advent of transistors and mainframes, rudimentary "monitor" programs appeared (like Fortran Monitor System - FMS, IBSYS). These weren't full OSes but could automate sequences of jobs (batches) provided on punched cards, loading and running programs one after another with minimal manual intervention. Still no multitasking or interaction during execution.
• The 1960s: Multiprogramming and Timesharing:
  • OS/360: Developed by IBM for its System/360 family, this was one of the first widely used operating systems. It introduced the concept of multiprogramming, where the CPU could execute one job while another job was waiting for slow I/O operations (like reading from tape). This required memory partitioning and basic protection to prevent jobs from interfering. OS/360 was notoriously complex and buggy.
  • MULTICS: (Multiplexed Information and Computing Service) An ambitious project aiming to allow hundreds of users to access a large computer simultaneously via terminals (teleprinters). It pioneered concepts like hierarchical file systems, protected memory segmentation, and timesharing (rapidly switching the CPU between multiple users' tasks to give the illusion of simultaneous access). While not commercially successful itself, it heavily influenced later systems.
  • UNIX: Developed at Bell Labs as a reaction to the complexity of MULTICS, initially for a single user. UNIX adopted key MULTICS concepts but with a simpler design. Its source code being available led to its wide adoption and evolution into many variants (like AT&T's System V and Berkeley Software Distribution - BSD). UNIX introduced powerful command-line tools and a simple, consistent file system structure. The POSIX standard later emerged to promote compatibility between different UNIX-like systems.
• The 1980s: Microcomputers and the Rise of Personal Computing:
  • The invention of large-scale integrated circuits enabled affordable personal computers (microcomputers).
  • CP/M (Control Program for Microcomputers): An early dominant OS for 8-bit microcomputers.
  • MS-DOS (Microsoft Disk Operating System): IBM needed an OS for its first PC. Microsoft bought a system called 86-DOS and adapted it. MS-DOS became the standard for IBM PCs and compatibles. It was a single-user, single-tasking, command-line based OS, simple but limited. Later versions incorporated some features seen in UNIX.
  • Early GUIs: Apple's Macintosh (introduced in 1984) popularized the Graphical User Interface (GUI), proving much more user-friendly than command-line systems for non-technical users. This shift fundamentally changed personal computing interaction.
• The 1990s - Present: Windows Dominance, UNIX/Linux in Servers, and Mobile OS Explosion:
  • Microsoft Windows: Initially a GUI "overlay" on MS-DOS, Windows evolved into a standalone OS (Windows 95 onwards, architecturally shifting towards NT). It became the dominant desktop OS. Its architecture borrowed heavily from systems like VAX/VMS in its kernel and component design.
  • Linux: A kernel initially developed by Linus Torvalds in 1991, inspired by MINIX (an educational UNIX variant). Distributed under the GNU GPL (General Public License), making its source code freely available. Linux distributions (kernel + system utilities) grew rapidly, becoming dominant in servers, supercomputers, and embedded systems due to their flexibility, stability, and open-source nature.
  • Mobile Operating Systems: The rise of smartphones led to new OS categories. Early players like Symbian OS and BlackBerry OS were overtaken by Apple's iOS (for iPhone, 2007) and Google's Android (based on the Linux kernel, 2008). These mobile OSes emphasize touch interfaces, power efficiency, and connectivity. Android became the most popular OS globally due to its widespread use on smartphones and embedded devices with screens.

This history shows a progression from direct hardware control to complex layers of abstraction and resource management, driven by changing hardware capabilities, user needs, and the desire to run multiple programs and support multiple users efficiently and securely.

Building Your Own: Operating System Development as a Hobby

For those interested in the "building from scratch" aspect, developing a hobby operating system is a direct way to learn how all these low-level components fit together.

Hobby Operating System: An operating system project developed primarily for personal learning, experimentation, or enjoyment, typically not derived directly from a major existing OS codebase and usually having a small user base and developer community.

Hobby OS development can take many forms:

• For "Homebrew" Hardware: Creating an OS for a simple computer system built from basic components (like a 6502 or Z80 based system). This involves writing very low-level code (often in assembly language) to interact directly with simple hardware.
• For Existing Architectures: Developing an OS that runs on common modern hardware platforms (like x86 or ARM), but starting the OS codebase from scratch or based on minimal educational kernels.
• Exploring New Concepts: Experimenting with novel OS designs, memory management techniques, or scheduling algorithms.

Developing an OS requires a deep understanding of computer architecture, memory management, concurrency, device interaction, and low-level programming. It's a significant undertaking but incredibly rewarding for understanding how computers truly work from the ground up. Examples of hobby OSes include Syllable, TempleOS, and many smaller projects often found on platforms like GitHub or dedicated OS development forums.

The Challenge of Compatibility: Diversity and Portability

The diversity of operating systems, while offering choices and specialized solutions, creates challenges for application developers. An application written for one OS (e.g., Windows) typically cannot run directly on another OS (e.g., Linux or macOS) without modification.

This is because:

• Different System Call Interfaces: OSes provide different sets of system calls, often with different names, parameters, and behaviors, for performing fundamental operations like file access, network communication, or process creation.
• Different Internal Structures: File system formats, memory management schemes, and process representations can vary widely.
• Different Hardware Interactions (via Drivers): Applications rely on the OS to talk to hardware, and the OS's device driver model and interface differ.

Porting an application from one OS to another requires adapting the parts of the code that interact with the OS, which can be costly and time-consuming.

To mitigate this, several approaches are used:

• Software Platforms/Middleware: Writing applications against higher-level software platforms or frameworks (like Java Virtual Machine, .NET, Qt, Electron). These platforms provide their own abstraction layer above the OS. The platform vendor bears the cost of porting the platform to various OSes, and applications written for the platform can then run on any OS where the platform is available.
• OS Standards and Abstraction Layers: Adopting industry standards for OS interfaces, such as POSIX (Portable Operating System Interface), which defines a standard set of APIs for Unix-like systems. Writing code that exclusively uses POSIX calls makes it much easier to port the application between different POSIX-compliant OSes. Another approach is using OS abstraction layers within the application code itself, where OS-specific calls are wrapped in a generic interface.

Examples in the Wild: Popular Operating Systems

Understanding the concepts discussed above provides a foundation for appreciating the design and features of popular operating systems encountered daily.

• Linux: A highly influential open-source, Unix-like operating system kernel. Linux distributions combine the kernel with system utilities (many from the GNU project), libraries, and application software.

  • Key Characteristics: Free and open-source (GPL), emphasizing simplicity, consistency, and modularity. Supports preemptive multitasking, multiple users, and a wide range of hardware. Uses a monolithic kernel architecture (though device drivers can often be loaded dynamically). Supports both command-line and graphical user interfaces (through various desktop environments).
  • Widespread Use: Dominant in servers, supercomputers, embedded systems, and mobile devices (via Android). Its flexibility allows it to run on systems from minimal embedded boards to massive data centers.
  • Android: The most popular OS globally by user base, primarily on smartphones and tablets. Android is built on a modified Linux kernel but uses a different software stack above it (libraries, application framework). While based on Linux, it's a distinct platform for application development (often using Java/Kotlin).

• Microsoft Windows: A proprietary operating system family developed by Microsoft.

  • Key Characteristics: Wide market share on desktop/laptop computers. Designed for compatibility, performance, and security. Uses a hybrid kernel architecture (Windows Executive) with many core services running in kernel mode. Supports preemptive multitasking, multiple users, advanced memory management (demand paging of virtual memory), and a sophisticated file system (NTFS).
  • Widespread Use: Dominant on personal computers, also used in servers (though less than Linux), workstations, and game consoles (Xbox).
  • Features: Includes a robust security model with access control lists, security descriptors for objects, and privilege management. Uses the Windows Driver Model (WDM) for device drivers. Provides a rich graphical user interface and supports various input methods.

These examples illustrate how the fundamental concepts of process management, memory allocation, file systems, security, and user interfaces are implemented and packaged in different ways to serve diverse computing needs.